Facility inspection system and facility inspection method

ABSTRACT

A facility inspection system prevents a normal part from being detected as an abnormal part caused by a deviation in an alignment due to a presence/absence of a moving object in detecting the abnormal part in a surrounding environment of a vehicle moving on a track. The system includes a photographing device, storage device, separation unit, an alignment unit, and a extraction unit. The photographing device photographs the surrounding environment of the moving vehicle. The storage device stores a reference alignment point cloud and a reference difference-extraction point cloud for each position on the track. The separation unit separates the alignment point cloud from a three-dimensional point cloud. The alignment unit aligns the reference alignment point cloud and the alignment point cloud and outputs alignment information. The extraction unit extracts a difference between the three-dimensional point cloud deformed based on the alignment information and the reference difference-extraction point cloud.

TECHNICAL FIELD

This disclosure relates to a facility inspection system and a facilityinspection method.

BACKGROUND ART

There are various railroad facilities such as a substation, a beam, andan overhead line around a railroad rail. When foreign objects adhere tothese facilities or damages are caused on the facilities, a railroadoperation is interfered. FIG. 16 illustrates examples of the adherenceof the foreign objects to the railroad facility and the damage of therailroad facility. When a flying object 15A such as a polyethylene bagand clothes adheres to the overhead line, a damage is caused on arailroad vehicle passed by. When a damage 15B occurs on a hanger thathangs the overhead line, the overhead line droops down to cause a damageon the railroad vehicle passed by and a failure of other railroadelectrical equipment due to a ground fault. When a part of a member thatsupports the beam and the overhead line drops off, a deformation, acollapse, or the like of this facility possibly brings danger to apassenger and the public. For these failures, currently, maintenanceinspectors actually inspect states around the railroad rail by visualchecks to detect presence/absence of the foreign object and the damage.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Unexamined Patent Application PublicationNo. 2010-202017

SUMMARY OF INVENTION Technical Problem

When the maintenance inspectors perform the visual checks of thefacilities around the railroad rails, the maintenance inspectorsactually move along the rails and confirm the absence of the adherenceof foreign objects and the damage on the railroad facilities. However,this inspection method causes the following problems as long as themaintenance inspector is involved. For example, (i) the inspectioncannot be frequently performed, thus unavoidably having an intervalbetween the inspections, (ii) mobilizing the maintenance inspectorscauses a considerable cost at every inspection, and (iii) the differencein ability and experience between the maintenance inspectors causes thedifference in the inspection result. Therefore, automated inspectionmethods of the railroad facility have been examined.

Patent Literature 1 discloses an obstacle detection system. In thesystem of Patent Literature 1, a three-dimensional sensor is installedto a vehicle, a three-dimensional point cloud in a surroundingenvironment is obtained while moving on the rail, and an obstacleentered in an operating vehicle area indicating an area of the vehicleis detected from the obtained three-dimensional point cloud. However,the system of Patent Literature 1 has a problem that while the foreignobject (for example, a plant) entered in the operating vehicle area canbe detected, the foreign object and the damage on the facility(hereinafter simply referred to as an “abnormal part”) outside theoperating vehicle area, for example, a flying object adhered to theoverhead line, a damage of the hanger, and a damage of a substationfacility, cannot be detected.

As a method for detecting the abnormal part outside the operatingvehicle area, there is a technique to identify the abnormal part bycomparing a current three-dimensional point cloud with a past normalthree-dimensional point cloud. In this technique, the barycentricposition of the past normal three-dimensional point cloud and thebarycentric position of the current three-dimensional point cloud arecalculated and an alignment is performed so as to eliminate thedifference between these barycentric positions. Then, the currentthree-dimensional point cloud is compared with a normalthree-dimensional point cloud after the alignment, and a point cloudindicating a difference is detected as the abnormal part.

However, in the above-described method, when the three-dimensional pointcloud includes a moving object, the alignment between the past normalthree-dimensional point cloud and the current three-dimensional pointcloud cannot be performed well. FIG. 17 illustrates a concrete exampleof the moving object around the rail. As illustrated in FIG. 17,examples of the moving object include a plant 16A grown near therailroad facility and swayed in a wind, an automobile 16B traveling on aroad adjacent to the railroad facility, and similar object. When thecurrent three-dimensional point cloud includes a three-dimensional pointcloud of the moving object as illustrated in FIG. 17, the barycentricposition of the current three-dimensional point cloud deviates comparedwith a three-dimensional point cloud without the moving object. In thiscase, a problem arises in that a deviation occurs in the alignmentbetween the barycentric position of the past normal three-dimensionalpoint cloud and the barycentric position of the currentthree-dimensional point cloud, resulting in detecting, as the abnormalpart, a normal part originally being normal.

Therefore, this disclosure provides a technique to prevent a normal partfrom being detected as an abnormal part caused by a deviation in analignment due to a presence/absence of a moving object in detecting theabnormal part in a surrounding environment of a vehicle moving on atrack (for example, a rail).

Solution to Problem

For example, to solve the above-described problems, a configurationdescribed in claims is employed. This application includes a pluralityof means to solve the above-described problems, and as one example, afacility inspection system is provided. The facility inspection systemincludes a photographing device, a storage device, an alignment areaseparation unit, an alignment unit, and a difference extraction unit.The photographing device photographs an image of a surroundingenvironment of a vehicle moving on a track. The storage device stores areference alignment point cloud and a reference difference-extractionpoint cloud for each position on the track. The alignment areaseparation unit separates an alignment point cloud from athree-dimensional point cloud obtained from the image. The alignmentunit performs an alignment of the reference alignment point cloud andthe alignment point cloud. The alignment unit outputs alignmentinformation. The difference extraction unit extracts a differencebetween the three-dimensional point cloud deformed based on thealignment information and the reference difference-extraction pointcloud.

With another example, a facility inspection method is provided. Thefacility inspection method includes a step of photographing an image ofa surrounding environment of a vehicle moving on a track by aphotographing device, a step of separating an alignment point cloud froma three-dimensional point cloud obtained from the image by an alignmentarea separation unit, a step of performing an alignment of a referencealignment point cloud stored in a storage device and the alignment pointcloud and outputting alignment information by an alignment unit, and astep of extracting a difference between the three-dimensional pointcloud deformed based on the alignment information and a referencedifference-extraction point cloud stored in the storage device by adifference extraction unit.

Advantageous Effects of Invention

This disclosure can prevent the normal part from being detected as theabnormal part caused by the deviation in the alignment due to thepresence/absence of the moving object in detecting the abnormal part inthe surrounding environment of the vehicle moving on the track (forexample, the rail). Further features pertain to this disclosure willbecome clear by descriptions in this Description and attached drawings.Problems, configurations, and effects other than ones described abovewill be made apparent from the following description of embodiments.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a drawing describing a configuration of a railroad facilityinspection system according to a first embodiment.

FIG. 2 is a drawing describing a configuration of an image processingdevice according to the first embodiment.

FIG. 3 is a drawing describing a point cloud DB.

FIG. 4 is a drawing describing a method for calculating coordinates toconvert from a range image to a three-dimensional point cloud.

FIG. 5 is a drawing illustrating an exemplary alignment area.

FIG. 6 is a drawing illustrating an exemplary alignment area.

FIG. 7 is a drawing illustrating an exemplary alignment area.

FIG. 8 illustrates an exemplary flowchart describing a process by analignment area separation unit according to the first embodiment.

FIG. 9 is a drawing describing a modification of the first embodiment.

FIG. 10 is a drawing describing a configuration of an image processingdevice according to a second embodiment.

FIG. 11 is a drawing describing an area DB.

FIG. 12 illustrates an exemplary narrowing process of alignment areacandidate information.

FIG. 13 illustrates an exemplary flowchart describing a process by analignment area separation unit according to the second embodiment.

FIG. 14 is a drawing describing a configuration of an image processingdevice according to a third embodiment.

FIG. 15 illustrates an exemplary flowchart describing a process by analignment area choosing unit according to the third embodiment.

FIG. 16 is a drawing describing adherence of a foreign object to therailroad facility and a damage of the railroad facility.

FIG. 17 illustrates a concrete example of a moving object around therail.

DESCRIPTION OF EMBODIMENTS

The following describes embodiments of this disclosure with reference tothe attached drawings. While the attached drawings illustrate specificembodiments based on principles of this disclosure, the drawings areprovided merely for understanding this disclosure, and are by no meansused for understanding this disclosure in a limited way.

The following embodiments relate to a technique to automatically detectan abnormal part (for example, a foreign object and a damage offacility) in a surrounding environment of a vehicle moving on a track.Here, the track includes, for example, a rail and a guide rail. Thevehicle includes various vehicles traveling on the track, for example, abusiness vehicle, a test vehicle, and a maintenance vehicle.

First Embodiment

FIG. 1 is a drawing describing a configuration of a railroad facilityinspection system according to the first embodiment. The railroadfacility inspection system includes a vehicle 100 and a ground station160.

The vehicle 100 includes a photographing device 110, an image processingdevice 120, a vehicle position identification device 130, a storage(storage device) 140, and a communication device 150. As describedabove, the vehicle 100 may be the test vehicle, a road railer, andsimilar vehicle in addition to the business vehicle insofar as thevehicle 100 can move along a railroad rail.

The photographing device 110 obtains a range image 200 of thesurrounding environment of the vehicle 100, and outputs the range image200 to the image processing device 120. The photographing device 110only needs to include a device configured to obtain the range image, forexample, a stereo camera and a three-dimensional sensor of aTime-of-Flight (TOF) method. For example, the photographing device 110has a detection distance of 0 to several tens of meters, for example, 0to 30 m.

The following description uses an example where the photographing device110 photographs an image of a forward range in a traveling direction ofthe vehicle 100. A photograph target of the photographing device 110 isnot limited to the forward range of the vehicle 100, but any range (forexample, a rear range in the traveling direction and a range in alateral direction perpendicular to the traveling direction) in alldirections of the vehicle 100 can be set to the photographed target byappropriately changing an installation position of the photographingdevice 110.

The image processing device 120 receives the range image 200 as inputinformation from the photographing device 110, receives vehicle positioninformation 210 as input information from the vehicle positionidentification device 130, and obtains reference three-dimensional pointcloud information 230 from the storage 140. The image processing device120 executes a changed area detection process based on these pieces ofinput information, and outputs changed area information 220 to thecommunication device 150. Here, the changed area is an area detected bycomparing a current three-dimensional point cloud with a past normalthree-dimensional point cloud, and this changed area can be consideredas an abnormal part (foreign object and damaged part).

The vehicle position identification device 130 uses, for example, aGlobal Positioning System (GPS) device and an Inertial Measurement Unit(IMU) to identify a current position and the like of the vehicle 100,and outputs them as the vehicle position information 210 to the imageprocessing device 120. The storage 140 holds information read by theimage processing device 120 as a database. In this embodiment, thecurrent position of the vehicle is identified with a latitude, alongitude, and an orientation, while the vehicle position information isnot limited to this. As the vehicle position information, a distance(kilometrage) from a starting point of the rail and information on aline (for example, up, down, main line, and subsidiary main line) may beused.

The communication device 150 receives the changed area information 220as the input information, and transmits it to the ground station 160 asa changed area detection signal 240. For communications between thecommunication device 150 and the ground station 160, a general line suchas a mobile phone network may be used, or another network may be used.The ground station 160 may include a processing device (not illustrated)configured to receive the changed area detection signal 240 transmittedfrom the communication device 150 of the vehicle 100 and present theabnormal part to a maintenance inspector via a display device (forexample, a display). The ground station 160 may include a documentgeneration device (not illustrated), and may automatically generate abusiness document such as a facility inspection report and a repair workinstruction by laying out information of the abnormal part, date/timeand a place where the abnormal part is found, and similar matter on thedocument via this document generation device.

FIG. 2 is a drawing illustrating a configuration of the image processingdevice according to the first embodiment. The image processing device120 includes a point cloud conversion unit 310, an alignment areaseparation unit 320, an alignment unit 330, a neighborhood retrievalunit 340, and a difference extraction unit 350.

The image processing device 120 may be achieved, for example, using ageneral use computer. The image processing device 120 may includeprocessing units each achieved as a function of a program executed onthe computer. The computer at least includes a processor such as aCentral Processing Unit (CPU) and a storage unit such as a memory. Theprocess by the image processing device 120 may be achieved such thatprogram codes corresponding to respective processing units are stored inthe memory and the processor executes each of the program codes.

A point cloud Database (DB) 300 is a database implemented to the storage140. FIG. 3 illustrates an exemplary point cloud DB 300. The point cloudDB 300 includes position information 500 and reference three-dimensionalpoint cloud information 540 as configuration items. The point cloud DB300 includes past point cloud information where normal states have beenconfirmed at respective positions (position information 500) on therail. In other words, the point cloud DB 300 is a DB that storesreference information relating the surrounding environment of thevehicle 100 where the normal state has been confirmed.

The reference three-dimensional point cloud information 540 includes areference alignment point cloud 510 and a referencedifference-extraction point cloud 520. The reference alignment pointcloud 510 is a point cloud used in an alignment process in the alignmentunit 330. The reference alignment point cloud 510 may be a point cloudobtained from images of respective positions on the rail by thephotographing device 110 at a travel on the rail in the past. Forexample, the reference alignment point cloud 510 includes point cloudsobtained at respective positions on the rail in an area 550 of FIG. 5when the vehicle 100 traveled on the rail in the past. The referencedifference-extraction point cloud 520 is a point cloud as a referencefor detecting the changed area. For example, the referencedifference-extraction point cloud 520 includes point clouds obtainedfrom the images of the respective positions on the rail by thephotographing device 110 when the vehicle 100 traveled on the rail inthe past. Thus, the point cloud DB 300 holds the reference alignmentpoint cloud 510 and the reference difference-extraction point cloud 520in association with the position information 500 of the photographedposition.

The position information 500 holds the respective positions on the railas the information of the latitude, the longitude, and the orientation.The reference alignment point cloud 510 and the referencedifference-extraction point cloud 520 each have a set of XYZcoordinates. As described above, as the position information 500, thedistance (kilometrage) from the starting point of the rail and theinformation on the line (for example, up, down, main line, andsubsidiary main line) may be used.

While the example where the vehicle 100 is actually traveled on the railto generate the point cloud DB 300 is described above, the configurationis not limited to this. The point cloud DB 300 may be generated fromdesign information on the rail and the surrounding environment (forexample, CAD data). With this configuration, the changed area can bedetected from the design information as the reference. When the pointcloud DB 300 is generated from the design information as the reference,the point cloud DB 300 is applicable to a use for confirming thesurrounding environment to be constructed as designed by causing thevehicle 100 to travel after the construction of the rail and itssurrounding environment. This is because the changed area information220 detected by the image processing device 120 indicates the differencewith the design information, thus ensuring the detection of a part notconstructed as designed.

The point cloud conversion unit 310 converts the range image 200 inputfrom the photographing device 110 into a three-dimensional point cloud400 and outputs the three-dimensional point cloud 400 to the alignmentarea separation unit 320. FIG. 4 is a drawing describing a method forcalculating coordinates to convert from the range image 200 to thethree-dimensional point cloud 400. The range image 200 has pixels eachobtained from a distance L to an object, and an angle φ on a ZY-planeand an angle θ on a XZ-plane from a center of an image. The point cloudconversion unit 310 performs a conversion process from the range image200 into coordinates of respective points of the three-dimensional pointcloud 400 based on Math. 1.

x=L cos ϕ sin θ

y=L sin ϕ

z=−L cos ϕ cos θ  [Math. 1]

The alignment area separation unit 320 receives the three-dimensionalpoint cloud 400 as the input information from the point cloud conversionunit 310. The alignment area separation unit 320 outputs the point cloudincluded in the alignment area in the three-dimensional point cloud 400as an alignment point cloud 410 to the alignment unit 330, and outputsevery point cloud in the three-dimensional point cloud 400 as adifference-extraction point cloud 420 to the difference extraction unit350. A parameter 360 defining the alignment area may be held in thestorage 140 for example, and the alignment area separation unit 320 canobtain the parameter 360 defining the alignment area from the storage140.

FIG. 5 illustrates an example of extracting the point cloud included inthe alignment area as the alignment point cloud. An alignment area 550is configured by a plurality of parameters, and for example, a spaceforward and upward of the vehicle 100. The space forward and upward ofthe vehicle 100 (the space obliquely upward with respect to the vehicle100) is an area mainly including, for example, an overhead line and aconstruction supporting the overhead line, and an area seemed to includefew moving objects. For example, the alignment area 550 is set at aposition forward of the vehicle 100 by Pf and a position having a heightPv from the rail position as a reference. The alignment area 550 isrepresented as a three-dimensional space having a height P_(H), a depthP_(D), and a width P_(W).

For example, the alignment area 550 is preferably configured to a spacelikely to include few moving objects such as a person moving in astation building, another train traveling on an adjacent rail, and avehicle traveling on an adjacent road. For example, when thephotographing device 110 is installed on a driving position, thealignment area 550 is preferably configured in a range of Pf=3 to 6 m,Pv=4 to 8 m, P_(H)=2 to 6 m, P_(W)=3 to 20 m, and P_(D)=10 to 70 m.

For example, the configuration of Pf=3 m, Pv=4 m, P_(H)=3 m, P_(W)=10 m,and P_(D)=70 m ensures specification of the alignment area 550 as thearea including few moving objects. These parameters may be configured tobe adjustable from outside according to a railroad standard and anoperation environment such as a vehicle height and a height of a sensorinstalled to the vehicle.

FIG. 6 illustrates an exemplary alignment area 550. The alignment area550 is preferably configured so as not to include the moving object andinclude an object with little change in state due to the environment andthe like. As one example, the alignment area 550 is configured so as toinclude at least a part of a utility pole 610 disposed along the railand at least a part of a beam 620 disposed between the utility poles610. Specifically, as illustrated in FIG. 6, the parameters (forexample, P_(W) and P_(H)) defining the alignment area 550 may beconfigured so as to include upper end portions of the utility poles 610and the beam 620.

FIG. 7 illustrates another example of the alignment area 550. From aperspective of not including the moving object and including the objectwith little change in state due to the environment and the like, thealignment area 550 may be configured so as to include a slope 710 on aside surface side of the vehicle 100. The slope is an inclined surfacepart disposed by cutting or piling up the ground when the rail is laid.

The above-described alignment areas 550 are merely examples, and theconfiguration is not limited to them. The photographing device 110 maybe disposed on a roof of the vehicle, and a space upward with respect tothe vehicle may be used as the alignment area 550. As another example,the alignment area 550 may be configured so as to include a constructionsuch as a building adjacent to the railroad rail, a roof of a platformof a station, and similar object.

FIG. 8 illustrates an exemplary flowchart describing a process by thealignment area separation unit 320. The alignment area separation unit320 performs an alignment area determination first with the points ofthe three-dimensional point cloud 400 input from the point cloudconversion unit 310 as a target point cloud (Step 800). The alignmentarea separation unit 320 determines whether the coordinate of the targetpoint cloud is present within the alignment area 550 or not.

When the target point is determined to be present within the alignmentarea 550 at Step 800, the alignment area separation unit 320 adds thispoint to the alignment point cloud 410 (Step 810), and then, the processproceeds to Step 820.

Meanwhile, when the target point is determined not to be present withinthe alignment area 550 at Step 800, the process proceeds to Step 820.

The alignment area separation unit 320 adds the target point to thedifference-extraction point cloud 420 (Step 820). After the flowchart ofFIG. 8 terminates on every point of the three-dimensional point cloud400, the alignment area separation unit 320 outputs the alignment pointcloud 410 to the alignment unit 330, and outputs thedifference-extraction point cloud 420 to the difference extraction unit350.

The neighborhood retrieval unit 340 receives the vehicle positioninformation 210 as the input information from the vehicle positionidentification device 130. The neighborhood retrieval unit 340 uses thevehicle position information 210 to retrieve information from the pointcloud DB 300, and reads the reference three-dimensional point cloudinformation 540 associated with the position information 500 nearest tothe vehicle position information 210. For example, in the case of thevehicle position information 210 (Nlat, Nlon), the neighborhoodretrieval unit 340 may determine the position information 500 (Rlat,Rlon) where an evaluation value V by Math. 2 becomes minimum as thenearest position information.

V=(Rlat−Nlat)²+(Rlon−Nlon)²   [Math. 2]

As another example, when the distance (kilometrage) from the startingpoint of the rail is used as the position information 500, theneighborhood retrieval unit 340 may determine the position information500 where a difference between a current kilometrage and a kilometrageof the position information 500 becomes minimum as the nearest positioninformation.

The neighborhood retrieval unit 340 outputs the reference alignmentpoint cloud 510 of the reference three-dimensional point cloudinformation 540 to the alignment unit 330, and outputs the referencedifference-extraction point cloud 520 of the reference three-dimensionalpoint cloud information 540 to the difference extraction unit 350.

The alignment unit 330 receives the alignment point cloud 410 as theinput information from the alignment area separation unit 320, andreceives the reference alignment point cloud 510 as the inputinformation from the neighborhood retrieval unit 340. The alignment unit330 performs the alignment of the alignment point cloud 410 with respectto the reference alignment point cloud 510. The alignment unit 330 mayemploy typically known Iterative Closest Point (ICP) method and NormalDistributions Transformation (NDT) method as the method of the pointcloud alignment. The alignment unit 330 rotates and translates thealignment point cloud 410 to obtain a rotational and translationalmovement parameter (amount of rotational and translational movement)where an error evaluation value with the reference alignment point cloud510 becomes minimum. Hereinafter, the rotational and translationalmovement parameter is referred to as “posture information 430.”

The alignment unit 330 outputs the obtained rotational and translationalmovement parameter to the difference extraction unit 350 as the postureinformation (alignment information) 430. A rotational and translationalmovement parameter A, a coordinate vector R of the reference alignmentpoint cloud 510, and a coordinate vector P of the correspondingalignment point cloud 410 have a relationship indicated by Math. 3.

R≈AP   [Math. 3]

The difference extraction unit 350 receives the difference-extractionpoint cloud 420 as the input information from the alignment areaseparation unit 320, receives the reference difference-extraction pointcloud 520 as the input information from the neighborhood retrieval unit340, and receives the posture information 430 as the input informationfrom the alignment unit 330.

The difference extraction unit 350 deforms the difference-extractionpoint cloud 420 based on the posture information 430. In detail, thedifference extraction unit 350 rotates and translates thedifference-extraction point cloud 420 based on the posture information430 to calculate a deformed three-dimensional point cloud. Thedifference extraction unit 350 calculates a distance of a nearest pointof the reference difference-extraction point cloud 520 for each point ofthe deformed three-dimensional point cloud (coordinate of each deformedpoint). The difference extraction unit 350 outputs a point having adistance equal to or more than a threshold value TL to the communicationdevice 150 as the changed area information 220.

Math. 4 indicates a determination of whether the points of thedifference-extraction point cloud 420 are each included in the changedarea information 220 or not assuming that the posture information 430 isA, the coordinate of each point of the difference-extraction point cloud420 is P, the coordinate of the point nearest to P of the referencedifference-extraction point cloud 520 is R, and the threshold value isTL.

length(R−AP)>TL difference

length(R−AP)≤TL not difference   [Math. 4]

The difference extraction unit 350 deforms the coordinates P of therespective points of the difference-extraction point cloud 420 inaccordance with the posture information A, and outputs the point of thedifference-extraction point cloud 420 as the changed area information220 when this point has the distance from the coordinate R of the pointnearest to P of the reference difference-extraction point cloud 520 islarger than the threshold value TL. Otherwise, the difference extractionunit 350 does not include that point of the difference-extraction pointcloud 420 in the changed area information 220. The difference extractionprocess described here is merely one example, and the differenceextraction process is not limited to this.

With this embodiment, the photographing device (for example, thethree-dimensional sensor) 110 mounted to the vehicle 100 obtains thethree-dimensional point cloud data around the vehicle. The alignmentarea separation unit 320 separates the point cloud in the spaceincluding few moving objects in the three-dimensional point cloud 400 asthe alignment point cloud 410. This ensures the alignment process in thespace including few moving objects. The alignment unit 330 can stablyperform the alignment between the past three-dimensional point cloud(reference alignment point cloud 510) and the current three-dimensionalpoint cloud (alignment point cloud 410). Consequently, the differenceextraction unit 350 obtains the difference between the past normalthree-dimensional point cloud (reference difference-extraction pointcloud 520) and the current three-dimensional point cloud(difference-extraction point cloud 420), thus detecting the abnormalpart such as the foreign objectoutside the operating vehicle area, andthe damage of the railroad-related facility.

Conventionally, there is a problem that a deviation occurs in thealignment between a barycentric position of the past normalthree-dimensional point cloud and a barycentric position of the currentthree-dimensional point cloud, resulting in a detection of the normalpart originally without the difference as the abnormal part. Incontrast, with this embodiment, the alignment area 550 where the areaincluding the moving object is eliminated as much as possible isconfigured and the point cloud for the alignment is extracted from thisalignment area 550. Therefore, this embodiment can prevent the normalpart from being detected as the abnormal part caused by a deviation inthe alignment due to the presence/absence of the moving object.Accordingly, the foreign object and the damage of the railroad-relatedfacility outside the operating vehicle area can be stably detected.

The above-described embodiment is merely one example and various changesare allowed to be made. For example, the difference extraction unit 350may store the changed area information 220 as the abnormalityinformation in the storage 140. When the vehicle 100 includes a displaydevice, the difference extraction unit 350 may output the changed areainformation 220 to the display device to notify an operator and the likeof the vehicle 100 of the foreign object and the damage of the facilityvia the display device.

For example, the photographing device 110 may further include a camerasensor that outputs a camera image (two-dimensional image), and theground station 160 may receive the two-dimensional image with thechanged area information 220. The processing device of the groundstation 160 may obtain a correspondence relationship between the pointsof the changed area (the three-dimensional point cloud as thedifference) extracted by the difference extraction unit 350 and therespective pixels of the two-dimensional image and superimpose theinformation (for example, marking with a frame and the like) indicativeof the abnormal part on the two-dimensional image using thiscorrespondence relationship. Then, the processing device of the groundstation 160 may display this two-dimensional image via the displaydevice. FIG. 9 includes an upper drawing illustrating one example wherethe information indicative of the abnormal part is displayed on thetwo-dimensional image. The display processing of the abnormal part isnot limited to such a marking, but may be performed with another methodensuring the abnormal part to be detected. This configuration ensurespresentation of the abnormal part with the two-dimensional image easilyseen by the user based on the obtained three-dimensional information onthe abnormal part.

The document generation device of the ground station 160 mayautomatically generate the business document such as the facilityinspection report or the repair work instruction by laying out theabove-described two-dimensional image and the information on thedate/time and a place where the abnormal part is found and similarmatter in accordance with a predetermined format. FIG. 9 includes alower drawing illustrating one example where the two-dimensional imageon which the abnormal part is indicated is arranged in the predeterminedformat to generate the business document. Various kinds of informationsuch as a time (date) and a position where the failure is detected and acreator of the business document may be input in an area other than thetwo-dimensional image. The processing device and the document generationdevice on the ground station 160 side may be mounted to the vehicle 100.

Second Embodiment

FIG. 10 is a drawing describing a configuration of an image processingdevice 120 according to the second embodiment. Identical referencenumerals are assigned for the components described in theabove-described first embodiment and the descriptions are omitted.

The image processing device 120 includes the point cloud conversion unit310, an alignment area separation unit 910, the alignment unit 330, theneighborhood retrieval unit 340, the difference extraction unit 350, andan area retrieval unit 930. An area DB 920 is a database implemented tothe storage 140.

FIG. 11 illustrates an exemplary area DB 920. The area DB 920 includesreference position information 1010 and alignment area candidateinformation 1020 as configuration items. The area DB 920 includes theinformation on the point clouds as candidates of the alignment area onrespective reference positions (reference position information 1010) onthe rail. The alignment area separation unit 320 of the first embodimentseparates the point cloud included in the predetermined space (alignmentarea 550) as the alignment point cloud 410. Meanwhile, the alignmentarea separation unit 910 of the second embodiment can change the spacefor extracting an alignment point cloud 950 corresponding to thealignment area candidate information 1020 of the area DB 920. The areaDB 920 holds the information on the appropriate alignment area candidatefor each position on the track (railroad rail). As one example, the areaDB 920 may hold an area (the utility pole and the beam) of FIG. 6 as theinformation on the alignment area candidate at one traveling positionwhile holding an area (the slope) of FIG. 7 as the information on thealignment area candidate at the other traveling position. Accordingly,the accuracy of the alignment process by the alignment unit 330improves, and consequently, the accuracy of the extraction of thechanged area information by the difference extraction unit 350 alsoimproves.

The area DB 920 holds the reference position information 1010 and thealignment area candidate information 1020 mutually matched. Thereference position information 1010 holds the information on thelatitude and the longitude indicative of the reference position and theorientation indicative of the reference direction. As the referenceposition information 1010, the distance (kilometrage) from the startingpoint of the rail and the information on the line (for example, up,down, main line, and subsidiary main line) may be used. The alignmentarea candidate information 1020 holds a plurality of areas including fewmoving objects in the surrounding environment of the railroad rail ascandidate areas of the alignment. The alignment area candidateinformation 1020 holds a set of XYZ coordinates of four apexes of atetrahedron as relative coordinates having the reference position andthe reference direction as the origin. While the alignment areacandidate information 1020 holds the information on the four apexes ofthe tetrahedron in this example, the configuration is not limited tothis and the information on any other space may be held.

The area retrieval unit 930 receives the vehicle position information210 as the input information from the vehicle position identificationdevice 130. The area retrieval unit 930 uses the vehicle positioninformation 210 to retrieve information from the area DB 920, and readsthe alignment area candidate information 1020 associated with thereference position information 1010 nearest to the vehicle positioninformation 210.

The area retrieval unit 930 may directly output the alignment areacandidate information 1020 as an alignment area 940 to the alignmentarea separation unit 910, while the alignment area candidate information1020 may be narrowed using a predetermined condition. The condition usedhere may include, for example, information on a measuring range of athree-dimensional sensor.

FIG. 12 illustrates an exemplary narrowing process of the alignment areacandidate information. Assume that the area retrieval unit 930 obtainsthe information on three alignment area candidates 1100, 1110, and 1120as the alignment area candidate information 1020 from the area DB 920.The area retrieval unit 930 reads the condition for the narrowing storedin the storage 140. The condition used here is the information on themeasuring range of the three-dimensional sensor, and includesinformation on a sensor proximity limit Lf and a sensor limit Ll aheadof the vehicle.

The area retrieval unit 930 sets a first sensor limit plane 1130 and asecond sensor limit plane 1140 based on the information on the sensorproximity limit Lf and the sensor limit Ll. The area retrieval unit 930extracts an area defined by the alignment area candidates 1100, 1110,and 1120 and included in a space between the first sensor limit plane1130 and the second sensor limit plane 1140 as the alignment area 940.The area retrieval unit 930 outputs the extracted alignment area 940 tothe alignment area separation unit 910. With this configuration, thearea retrieval unit 930 narrows the alignment area candidate accordingto the condition and outputs it to the alignment area separation unit910, thus reducing the amount of calculation in the process performedthereafter. Consequently, the speed of the separation process of thepoint cloud by the alignment area separation unit 910 and the speed ofthe alignment process by the alignment unit 330 can be improved.

The alignment area candidate (tetrahedron) 1100 is present forward withrespect to the first sensor limit plane 1130 in the traveling directionof the vehicle 100, thus not being included in the alignment area 940.Meanwhile, the alignment area candidate (tetrahedron) 1110 has a part ofthe tetrahedron present forward with respect to the first sensor limitplane 1130. Then this part is not included in the alignment area 940 andthe other part is included in the alignment area 940. The alignment areacandidate (tetrahedron) 1120 is present in the space between the firstsensor limit plane 1130 and the second sensor limit plane 1140. Then,the whole alignment area candidate 1120 is included in the alignmentarea 940.

The alignment area separation unit 910 receives the three-dimensionalpoint cloud 400 as the input information from the point cloud conversionunit 310, and receives the alignment area 940 corresponding to thecurrent position of the vehicle 100 as the input information. Thealignment area separation unit 910 outputs the point cloud included inthe alignment area 940 in the three-dimensional point cloud 400 as thealignment point cloud 950 to the alignment unit 330, and outputs everypoint cloud in the three-dimensional point cloud 400 as thedifference-extraction point cloud 420 to the difference extraction unit350.

FIG. 13 illustrates an exemplary flowchart describing the process by thealignment area separation unit 910. The alignment area separation unit910 performs an alignment area determination first with the points ofthe three-dimensional point cloud 400 input from the point cloudconversion unit 310 as a target point cloud (Step 1200). The alignmentarea separation unit 910 determines whether the coordinate of the targetpoint cloud is present within the alignment area 940 output from thearea retrieval unit 930 or not.

When the target point is determined to be present within the alignmentarea 940 at Step 1200, the alignment area separation unit 910 adds thispoint to the alignment point cloud 950 (Step 1210), and then, theprocess proceeds to Step 1220.

Meanwhile, when the target point is determined not to be present withinthe alignment area 940 at Step 1200, the process proceeds to Step 1220.

The alignment area separation unit 910 adds the target point to thedifference-extraction point cloud 420 (Step 1220). After the flowchartof FIG. 13 terminates on every point of the three-dimensional pointcloud 400, the alignment area separation unit 910 outputs the alignmentpoint cloud 950 to the alignment unit 330, and outputs thedifference-extraction point cloud 420 to the difference extraction unit350.

The reference alignment point cloud 510 of the point cloud DB 300 ofthis embodiment holds the past point cloud corresponding to thealignment area 940. The alignment unit 330 performs the alignment of thealignment point cloud 950 with respect to the reference alignment pointcloud 510. The process of the neighborhood retrieval unit 340, theprocess of the difference extraction unit 350, and similar process aresimilar to those of the first embodiment.

With this embodiment, the area retrieval unit 930 retrieves the knownperipheral alignment area candidate information 1020 from the area DB920 based on the vehicle position information 210 in the vehicleposition identification device 130. The alignment area separation unit910 can extract the alignment point cloud 950 from the three-dimensionalpoint cloud 400 based on the alignment area candidate information 1020.With this embodiment, the alignment area can be flexibly specified foreach vehicle position compared with the first embodiment. For example,in the example of FIG. 5, the space for extracting the alignment pointcloud is the limited space forward and upward of the vehicle 100 at anyposition on the rail. However, in this embodiment, the area obviouslywithout the moving object, for example, the construction such as abuilding adjacent to the railroad rail, also can be used as thealignment area for each traveling position. Especially, since thesurrounding environment of the railroad rail changes by the positions ofthe vehicle 100, the alignment area appropriate for each position can bespecified, thus ensuring the stable alignment process and differenceextraction process of the three-dimensional point cloud.

The above-described embodiment is merely one example and various changesare allowed to be made. For example, the facility inspection system mayinclude an interface and a processing unit (DB modification unit) formodifying the area DB 920. For example, when the user confirms thechanged area information 220 output from the difference extraction unit350 on the display device and the changed area information 220 includesmany errors, it is considered a case where the alignment area is notappropriately specified. The user may input the modified information viathe predetermined interface on the display device to cause the DBmodification unit to modify the alignment area candidate information1020 in the area DB 920 based on the input modified information.

Third Embodiment

FIG. 14 is a drawing describing a configuration of an image processingdevice 120 according to the third embodiment. Identical referencenumerals are assigned for the components described in theabove-described first embodiment and the descriptions are omitted.

The image processing device 120 further includes an alignment areachoosing unit 1310 in addition to the configuration of FIG. 2. Thealignment area choosing unit 1310 receives the three-dimensional pointcloud 400 as the input information from the point cloud conversion unit310, and obtains candidate area information 1300 stored in the storage140 as the input information. The candidate area information 1300includes the information on a plurality of candidate areas andinformation on priority orders of the respective candidate areas. Thealignment area choosing unit 1310 determines a count ofthree-dimensional points in each candidate area of the three-dimensionalpoint cloud 400 detected via the photographing device 110 in accordancewith the priority order, and chooses the area having a predeterminedcount or more of three-dimensional points as the alignment area.

FIG. 15 illustrates an exemplary flowchart describing a process by thealignment area choosing unit 1310. Here, as one example, the candidatearea information 1300 includes information on a first area as a firstpriority order and information on a second area as a second priorityorder. For example, the first area is an area including the utility pole610 and the beam 620 illustrated in FIG. 6, and the second area is anarea on the side surface side of the vehicle 100 illustrated in FIG. 7.The candidate area information 1300 may hold the information on thethree or more areas.

The alignment area choosing unit 1310 obtains the candidate areainformation 1300. Then, the candidate area choosing unit 1310 firstdetermines whether the count of three-dimensional points in the firstarea of the three-dimensional point cloud 400 detected via thephotographing device 110 is equal to or more than a threshold Th or not(Step 1400).

When the count of three-dimensional points in the first area is equal toor more than the threshold Th at Step 1400, the process proceeds to Step1420. The alignment area choosing unit 1310 determines the first area asthe alignment area (Step 1420). Then, the alignment area choosing unit1310 outputs alignment area information 1320 to the alignment areaseparation unit 320.

When the count of three-dimensional points in the first area is lessthan the threshold Th at Step 1400, the process proceeds to Step 1410.The alignment area choosing unit 1310 changes the alignment area to thesecond area as the second priority order (Step 1410). Then, the processreturns to Step 1400, and when the condition of Step 1400 is satisfied,the alignment area choosing unit 1310 determines the second area as thealignment area (Step 1420). When three or more candidate areas areconfigured, it is only necessary to repeatedly execute Steps 1400 and1410 by the count to determine the alignment area.

With this embodiment, the alignment area choosing unit 1310 can choosethe alignment area corresponding to the count of three-dimensionalpoints in the alignment area candidate detected via the photographingdevice 110 at that time. For example, the utility pole 610 and the beam620 illustrated in FIG. 6 are disposed along the rail at intervals ofabout 30 to 65 m. Accordingly, a situation where the utility pole 610and the beam 620 are not included in the measuring range of thephotographing device 110 occurs during the travel of the vehicle 100. Inthis case, the absence of the point cloud in the first area (areaforward and upward of the vehicle 100) fails in performing the alignmentwell. In this embodiment, even in such a case, the alignment areachoosing unit 1310 chooses the second area (area on the side surfaceside of the vehicle 100) as the alignment area, and determines thesecond area as the alignment area when the count of three-dimensionalpoints in the second area is equal to or more than the predeterminedcount. Thus, with this embodiment, the area where the three-dimensionalpoint cloud is detected can be chosen as the alignment area.

According to this embodiment, the plurality of candidate areas have thepriority orders. The alignment area choosing unit 1310 can determine thearea where the priority order is high and the count of three-dimensionalpoints is equal to or more than the predetermined count as the alignmentarea.

In this embodiment, it is only necessary that the reference alignmentpoint cloud 510 of the point cloud DB 300 holds the informationcorresponding to the area included in the candidate area information1300. Other processes are similar to those of the first embodiment.

This embodiment is also applicable to the second embodiment. Thealignment area choosing unit 1310 may receive the plurality of alignmentareas as the input information from the area retrieval unit 930 anddetermine the area having the predetermined count or more ofthree-dimensional points among the plurality of alignment areas as thealignment area. The alignment area choosing unit 1310 only needs tooutput the information on the determined alignment area to the alignmentarea separation unit 910.

This disclosure is not limited to the above-described embodiments butincludes various modifications. The above-described embodiments havebeen described in detail for easy understanding of this disclosure, andtherefore, it is not necessarily limited to include all describedconfigurations. It is possible to replace a part of the configuration ofone embodiment with a configuration of another embodiment. It ispossible to add a configuration of one embodiment to a configuration ofanother embodiment. Some of the configurations of each embodiment can beadded to, removed from, or replaced by another configuration.

While the neighborhood retrieval unit 340 obtains the referencethree-dimensional point cloud information 540 from the point cloud DB300 based on the current position information in the above-describedembodiment, the configuration is not limited to this. For example, amethod where numbers are assigned to the reference three-dimensionalpoint cloud information 540 for each position and the referencethree-dimensional point cloud information 540 is obtained in associationwith the number, or a method where tags are disposed on the rail and thereference three-dimensional point cloud information 540 of the identicaltag is obtained may be employed.

The respective configurations, functions, and the like of theabove-described image processing device 120 may be achieved by theprocessor configured to interpret and execute the program to achieveeach function with software. The information on the programs, files, andthe like to achieve respective functions can be stored in a storagedevice such as a memory, a hard disk, and a Solid State Drive (SSD) or arecording medium such as an IC card, an SD card, and a DVD. A part of orall the configurations and the like of the above-described imageprocessing device 120 may be achieved by hardware designed as, forexample, an integrated circuit.

In the above-described embodiments, control lines and information linesconsidered necessary for the explanation are described. All of thecontrol lines and the information lines of the product are notnecessarily described. All configurations may be mutually coupled.

LIST OF REFERENCE NUMERALS

-   100 vehicle-   110 photographing device-   120 image processing device-   130 vehicle position identification device-   140 storage (storage device)-   150 communication device-   160 ground station-   200 range image-   210 vehicle position information-   220 changed area information-   230 reference three-dimensional point cloud information-   240 changed area detection signal-   300 point cloud DB-   310 point cloud conversion unit-   320 alignment area separation unit-   330 alignment unit-   340 neighborhood retrieval unit-   350 difference extraction unit-   400 three-dimensional point cloud-   410 alignment point cloud-   420 difference-extraction point cloud-   430 posture information (alignment information)-   500 position information-   510 reference alignment point cloud-   520 reference difference-extraction point cloud-   540 reference three-dimensional point cloud information-   910 alignment area separation unit-   920 area DB-   930 area retrieval unit-   940 alignment area-   950 alignment point cloud-   1010 reference position information-   1020 alignment area candidate information-   1310 alignment area choosing unit

1. A facility inspection system comprising: a photographing device thatphotographs an image of a surrounding environment of a vehicle moving ona track; a storage device that stores a reference alignment point cloudand a reference difference-extraction point cloud for each position onthe track; an alignment area separation unit that separates an alignmentpoint cloud from a three-dimensional point cloud obtained from theimage; an alignment unit that performs an alignment of the referencealignment point cloud and the alignment point cloud, the alignment unitoutputting alignment information; and a difference extraction unit thatextracts a difference between the three-dimensional point cloud deformedbased on the alignment information and the referencedifference-extraction point cloud.
 2. The facility inspection systemaccording to claim 1, wherein the alignment area separation unit outputsa point cloud in the three-dimensional point cloud as the alignmentpoint cloud, and the point cloud is present in a predetermined spaceincluding few moving objects.
 3. The facility inspection systemaccording to claim 2, wherein the predetermined space is a spaceobliquely upward, upward, or on a side surface side of the vehicle. 4.The facility inspection system according to claim 3, wherein thepredetermined space is configured so as to include: a part of a utilitypole disposed along the track and a beam between the utility poles; or apart of a slope on the side surface side of the vehicle.
 5. The facilityinspection system according to claim 1, wherein the storage devicestores alignment area candidate information for each position on thetrack, and the alignment area separation unit outputs a point cloud inthe three-dimensional point cloud as the alignment point cloud, thepoint cloud being included in the alignment area candidate information.6. The facility inspection system according to claim 5, furthercomprising an area retrieval unit that retrieves the alignment areacandidate information from the storage device based on a currentposition of the vehicle.
 7. The facility inspection system according toclaim 6, wherein the area retrieval unit narrows the alignment areacandidate information based on a measuring range of the photographingdevice.
 8. The facility inspection system according to claim 1, furthercomprising a neighborhood retrieval unit that retrieves the referencealignment point cloud and the reference difference-extraction pointcloud from the storage device based on a current position of thevehicle.
 9. The facility inspection system according to claim 1, furthercomprising an alignment area choosing unit that determines an area amonga plurality of candidate areas as an alignment area, the area includinga predetermined count or more of three-dimensional points detected viathe photographing device.
 10. The facility inspection system accordingto claim 9, wherein the alignment area choosing unit chooses theplurality of candidate areas in accordance with priority orders.
 11. Afacility inspection method comprising: a step of photographing an imageof a surrounding environment of a vehicle moving on a track by aphotographing device; a step of separating an alignment point cloud froma three-dimensional point cloud obtained from the image by an alignmentarea separation unit; a step of performing an alignment of a referencealignment point cloud stored in a storage device and the alignment pointcloud and outputting alignment information by an alignment unit; and astep of extracting a difference between the three-dimensional pointcloud deformed based on the alignment information and a referencedifference-extraction point cloud stored in the storage device by adifference extraction unit.